Repository navigation
Conversation
…ore jep (#54) On Linux, libjep.so failed to load with "libpython3.X.so.1.0: cannot open shared object file" for a Python installed by uv, pyenv, conda or in a custom prefix, unless its lib directory was on LD_LIBRARY_PATH at JVM start. The environment probe now reports sysconfig LIBDIR and INSTSONAME; before Jep loads its native library, PythonEnvironment System.loads LIBDIR/INSTSONAME by its path (Linux only, shared library only), so the loader resolves libjep's dependency by soname. As the JVM loads it RTLD_LOCAL, and Jep's own RTLD_GLOBAL promotion looks it up by file name, which the loader cannot find, the first interpreter makes its symbols global by loading it as an extension module with RTLD_GLOBAL dlopen flags, so that C extensions not linked to libpython (numpy, _ctypes of most builds) still import. If loading still fails, the error names the directory to put on LD_LIBRARY_PATH. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
…54) Making libpython global only in the first interpreter came after the startup of CPython, so a C extension imported by a .pth file or sitecustomize, e.g. numpy or coverage, still failed with an undefined symbol. On Java 22 or newer, libpython is now loaded with dlopen(RTLD_GLOBAL) through the foreign linker (via reflection, the library targets Java 17) before Jep starts CPython. On Java 17 to 21 the System.load + make-global-in-the-first- interpreter fallback stays, now best-effort (logged, never failing startup), run by EmbeddedPythonStarter; README tells its .pth limitation. The LD_LIBRARY_PATH advice now walks the cause chain, also of Jep's retry Error, and is given only for the soname of the shared libpython found. PythonLibraryTest compiles a C extension not linked to libpython, imported by the code, by sitecustomize at startup, and without the foreign linker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
A system Python, whose libpython the loader finds by its soname, is left alone again, so it gets no new restricted method warning on Java 22+: loaderFinds checks LD_LIBRARY_PATH, /etc/ld.so.cache and the system directories for the same file, like the loader does. On Java 21, the foreign linker is no longer touched (allocateFrom is looked up before any restricted call), so it prints no warning either. The startup-import test of PythonLibraryTest is assumed on Java 22+ only, as on Java 17-21 libpython becomes global after the startup, as documented. Making libpython global on Java 17-21 is now synchronized, so that a second interpreter waits for it; the LD_LIBRARY_PATH advice is given only for "cannot open shared object file" / "Library not loaded" of the soname. README no longer lists conda as handled (its libjep has a RUNPATH), mentions RPATH, and DEVELOPMENT.md the pyenv LD_LIBRARY_PATH need on 17-21. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
…54) - A RuntimeException of System.load of libpython, e.g. IllegalCallerException with --illegal-native-access=deny, is kept as the preload failure instead of escaping initialize() unwrapped. - A failed preload is named in the startup failure message, instead of the LD_LIBRARY_PATH advice, which would not help then. - The noForeignLinker test seam is parsed as a boolean. - /etc/ld.so.cache is read only when no LD_LIBRARY_PATH entry decides. - The CLAUDE.md entry is trimmed to the non-obvious lesson. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
This was referenced Nov 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #54
On Linux, a Python whose
libpythonlies outside the system library directories (uv, pyenv--enable-shared, a custom prefix) now starts withoutLD_LIBRARY_PATH, wherelibjep.sofailed withUnsatisfiedLinkError: libpython3.X.so.1.0: cannot open shared object file.sysconfig'sLIBDIRandINSTSONAME.libpythonby its soname (loaderFindsimitatesLD_LIBRARY_PATH,/etc/ld.so.cacheand the default directories), it is loaded before Jep, with its symbols made global (C extensions not linked to libpython, like numpy, need them):dlopen(RTLD_GLOBAL)through the foreign linker (by reflection, the library targets Java 17) — C extensions imported at startup (.pth,sitecustomize) work too;System.load, then the symbols are made global in the first interpreter (best-effort, synchronized) — a C extension imported at startup still needsLD_LIBRARY_PATHthere, documented.libpython, the startup error names the directory to put onLD_LIBRARY_PATH(also through Jep's "previously failed to initialize" retry); a failure of the preload itself is reported as such.Review
3 rounds:
.pth/sitecustomize) still failed → foreign-linkerdlopenon Java 22+.ld.so.cache).Not tested here: pyenv
--enable-shared,actions/setup-pythonwithoutLD_LIBRARY_PATH(CI's Python matrix now removes it inPythonLibraryTest), conda, macOS.Follow-ups
libpythonof a relocated Python (staleLIBDIR) not found.libpythonnot global without thelibpython3.X.sodev symlink (pre-existing).#66 will be stacked on this branch — merge this PR first.
🤖 Generated with Claude Code
https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG